1. Field of the Invention
The present invention relates to a moving picture compression technique, and particularly to a method and apparatus for compression-encoding a moving picture supplied in real time after dividing it in units of a plurality of frames.
2. Description of the Related Art
There have been proposed several conventional moving picture compressing apparatuses for compression-encoding a moving picture supplied in real time as disclosed, for example, in Japanese Patent Application Laid-open No. 7-177514, No. 8-307878, No. 8-336138, and No. 10-136380. The details of these publications will be described.
A moving picture compressing apparatus disclosed in the above Publication No. 7-177514 is arranged such that frame decimation is performed during compression encoding of a moving picture and the compression-encoded data is transmitted to a receiver. At the receiver, the compression-encoded data is decoded and expanded to reproduce the original moving picture by interpolating frames decimated in the compression encoding. The interpolation is performed such that the frames to be interpolated are obtained by approximating the motion of a motion vector calculated from the adjacent frames to a curve representation and determining a ratio of the distances from a frame to be interpolated to adjacent frames. In the above arrangement, when a moving picture including a curved movement such as a circular or parabola motion is received, its frames subtracted for decimation at a transmitter can precisely be restored. Hereafter, this moving picture compressing apparatus disclosed in the above Publication No. 7-177514 is referred to as a first prior art.
An image-coding device disclosed in the above Publication No. 8-307878 is arranged such that the compression-encoding of a moving picture is performed with the use of intra-frame encoding mode and inter-frame encoding mode with motion compensation prediction. More specifically, a frame to be compression-encoded in the intra-frame encoding mode is compression-encoded in the inter-frame encoding mode when a difference between the generated code estimated amount of the frame of interest and the estimated amount of the past frame is small to the extent that its scene has small movements, and the frame of interest is not a scene change frame. In the above arrangement, when a moving picture includes a succession of still pictures, it can be compression-encoded at a higher quality. Hereafter, this image-coding device disclosed in the above Publication No. 8-307878 is referred to as a second prior art.
A moving image compression device disclosed in the Publication No. 8-336138 is arranged such that a moving picture is divided in units of a group of pictures (GOP) consisting of a plurality of frames before compression encoding. The number of frames which constitute one group of pictures GOP of interest is determined depending on the time code indicating a drop frame, which is predicted and detected from the time codes each indicating the stamped times attached to the frames.
More specifically, a group of pictures GOP is a group of frames for enabling random accessing or trick action modes including fast forwarding, rewinding, playback from midway, and reverse playback. In the NTSC (National Television System Committee) system which is employed for television broadcasting in Japan and U.S.A., the number of frames per second is 29.97 at maximum and the group of pictures GOP thus consists of approximately 30 frames for a length of one second. Since the number of frames in the group of pictures GOP is not a neat number, editing the frames e.g. of a broadcasting program in a duration of an integer number of minutes or seconds develops partially not continuous movements in a moving picture. For avoiding such a drawback, the frames should be decimated to a desired number and subtracted frames by the decimation are called dropped frames. Each frame is identified with a time code comprising four numbers: hours, minutes, seconds, and frame number. It is determined by prediction and identification whether or not the time codes assigned to dropped frames are received. When a dropped frame is found, the number of frames in the group of pictures GOP is changed. In the above arrangement, an integer number of minutes or seconds in the time code can be matched with the boundary of groups of pictures GOPs hence enabling the post editing of a signal (of bit stream) produced by the compression encoding with ease. This moving image compression device is referred to as a third prior art hereinafter.
A moving image encoding device disclosed in the above Publication No. 10-136380 is arranged such that compression-encoded data can be received at a constant frame rate even when an input picture cannot be received at a constant frame rate. More specifically, in the case of a loss of frame of data which is to be continuously inputted, the necessary encoded data is produced by copying previous or subsequent frames of data or by sending information indicating that it is the same image as the previous or subsequent frame. This moving image encoding device is referred to as a fourth prior art hereinafter.
A variety of moving picture compressing methods have been proposed depending on applications and characteristics of transmission lines over which moving picture data are transmitted. Characterized examples of such a moving picture compressing method with the use of CD-ROMs, DATs, (digital audio tapes), and hard disks as recording mediums are MPEG (Moving picture Experts Group) 1 and its new version, MPEG2, which are developed for data storage and standardized by ISO (International Organization for Standardization). The second and third prior arts described above both conform to the MPEG standards. In the MPEG standards, the number of frames per second to be received is predetermined (hence referred to as a frame rate hereinafter).
However, when the frame rate which is predetermined is changed due to a loss of some of the frames to be received by dropping or any other reason, not by intentional decimation of the frames such as schemed with the first or third prior arts, if it is used without correction in the compression encoding, then there will be developed a time discrepancy such as a delay in the time information at the receiver or playback apparatus. The drawback like this may appear in such a moving picture compressing apparatus as composed of a software architecture when a series of moving pictures are continuously received in real time from a source such as a video camera and subjected in a succession to the compression encoding. The low processing speed of a CPU (central processing unit) or a heavy load on the CPU due to multitask processing in parallel causes the speed of receiving the picture data to be decreased or prevents the picture data from being received at a constant speed.
The first prior art allows the frames decimated intentionally at the transmitter to be restored by interpolation at the receiver but has no scheme for compensating frames dropped with no intention at the transmitter, hence hardly accommodating changes in the frame rate. The second prior art also has no scheme for restoring dropped frames before the compression encoding, hence rarely making up for a change in the frame rate.
In the advanced moving picture compressing apparatus composed of a software architecture, its CPU is heavily loaded and takes more time to carry out the compression encoding of frames with the speed of receiving a moving picture being retarded, hence making the compression encoding of the moving picture in real time more difficult.
The third prior art allows the number of frames in an group of pictures GOP to be modified, when frames decimated intentionally (dropped frames) are found before the compression encoding, and thereby the right end of minutes or seconds of a time code is matched with the boundary of the group of pictures GOP. Since the remaining of the frames not subtracted is normally subjected to the compression encoding, reduction of the amount of data to be processed or minimizing the overall time required for the compression encoding will be difficult.
The fourth prior art sends information indicating that a dropped frame is the same image as the previous or subsequent frame to produce the necessary encoded data. Therefore, in the case of predicative encoding, a time distance between key frames (I-pictures) or sub-key frames (P-frames) becomes too long, resulting in remarkably reduced encoding efficiency and thereby deteriorated quality of image. Further, the dropped frame is nothing but B-frame because the dropped frame cannot be a reference frame. Therefore, the larger the number of dropped frames, the larger the number of consecutive B-frames inserted, causing the position of P-frame to be shifted backward.
It is thus an object of the present invention to provide a moving-picture compressing method and apparatus allowing discrepancy such as a delay in time information caused by a change in frame rate to be avoided, the amount of data to be processed to be reduced, and the overall time required for carrying out the compression encoding to be minimized.
According to the present invention, a method for compression-encoding a moving picture supplied in real time, wherein the moving picture is divided in units of a group of pictures consisting of a plurality of frames, comprises the steps of:
a) determining whether a predetermined number of frames required for forming a group of pictures have been inputted;
b) when the predetermined number of frames have not been all supplied, compensating for a dropped frame by interpolation of a proceeding frame previous to the dropped frame to produce an interpolated group of picture; and
c) compression-encoding each frame of the interpolated group of pictures to produce encoded moving picture data.
In the step c), an interpolated frame is preferably represented by information indicating that it is identical to the proceeding frame.
According to an aspect of the present invention, a method for compression-encoding a moving picture supplied in real time, wherein the moving picture is divided in units of a group of pictures consisting of a predetermined number of frames, comprises the steps of:
a) adding a frame number to each frame of the moving picture based on time information to each frame and a predetermined frame rate;
b) determining whether the predetermined number of frames required for forming a group of pictures have been all inputted, by referring to a frame number of each frame;
c) when the predetermined number of frames have not been all inputted, compensating for a dropped frame by interpolation of a proceeding frame previous to the dropped frame to produce an interpolated group of pictures; and
d) compression-encoding each frame of the interpolated group of pictures to produce encoded moving picture data to be stored.
According to another aspect of the present invention, a method for compression-encoding a moving picture supplied in real time, wherein the moving picture is divided in units of a group of pictures consisting of a predetermined number of frames, comprises the steps of:
a) determining whether a current frame is a head of a group of pictures;
b) when the current frame is the head of the group of pictures, obtaining an actual frame rate measured in a previous group of pictures;
c) modifying a structure of the group of pictures depending on a combination of the actual frame rate and a predetermined frame rate;
d) adding a frame number to each frame of the moving picture based on time information for each frame and the predetermined frame rate;
e) determining whether the predetermined number of frames required for forming a group of pictures have been all inputted, by referring to a frame number of each frame;
f) when the predetermined number of frames have not been all inputted, compensating for a dropped frame by interpolation of a proceeding frame previous to the dropped frame to produce an interpolated group of pictures; and
g) compression-encoding each frame of the interpolated group of pictures to produce encoded moving picture data to be stored.
A type of each frame of a group of pictures is one of Intra type, Predictive type, and Bidirectionally predictive type. In the step c), the structure of the group of pictures is preferably modified such that an interpolated frame is of Bidirectionally predictive type. In the step g), an interpolated frame is preferably represented by information indicating that it is identical to the proceeding frame.
According to the present invention, the arrangement of the present invention permits the data to be compression-encoded after its dropped frames have been interpolated. Therefore, even if the frame rate is changed, any discrepancy such as a delay in the time information which may be caused by the change in the frame rate can be prevented.
Also, according to the present invention, a dropped frame is restored by copying the preceding frame and thus the compression encoding of the present frame is not needed hence decreasing the overall processing time. In addition, the amount of data which indicates the identicalness of the frames is small and the overall amount of the data will successfully be reduced. This allows the compression encoding of the frames to be carried out at a higher speed and the amount of data saved by the reduction can be used for the other frames, hence ensuring the compression encoding of a high picture quality.
According to the other aspect of the present invention, the structure of an group of pictures is dynamically modified in response to the actual frame rate, hence decreasing the amount of the data to be processed and increasing the efficiency and speed of the compression processing to minimize the overall time required for the compression encoding.